Integration of hearing aids with smart glasses to improve intelligibility in noise

ABSTRACT

Disclosed herein, among other things, are methods and apparatuses for integration of hearing aids with an MSD to improve intelligibility in noisy environments. One aspect of the present subject matter relates to a method of providing voice audio to a hearing aid or text in a hearing aid user&#39;s field of view, where the voice audio or text corresponds to a speaker of interest within a noisy environment, and where the speaker of interest is identified using smart glasses. An MSD graphical user interface is provided for a hearing aid user to identify a speaker of interest. The lip movement patterns of the speaker of interest are recorded by the smart glasses, and voice activity detection is performed on the lip movement patterns. Noise reduction is performed using the results of the voice activity detection. The noise-reduced voice audio may be transmitted to the hearing aid, or the speech recognition text may be transmitted to and displayed on the MSD.

INCORPORATION BY REFERENCE

This application is related to U.S. Provisional Patent Application Ser.No. 61/826,483, filed on May 22, 2013, which is incorporated herein byreference in its entirety.

FIELD OF THE INVENTION

The present subject matter relates generally to hearing assistancedevices, and in particular to the integration of hearing aids with smartglasses to improve intelligibility in noise.

BACKGROUND

Modern “smart glasses” or augmented reality multisensory display devices(MSDs), such as Google Glass, may provide visual and auditoryinformation to the wearer of the device beyond what they normally wouldsee or hear without the device. MSDs may refer to any device thatprovides a heads-up display (e.g., a display in the current field ofview) of visual information from an MSD or other computing device. MSDsmay also provide the capability to recognize input speech. The speechrecognition capability may be built-in, or may be enabled by sending andreceiving recorded audio or speech recognition results to or from anetwork (e.g., the internet, the “cloud”) or another computing device.In various MSDs, the sensory information provided to the wearer of thedevice includes an augmentation of the environment around the wearer(visually or aurally displaying identification of objects around thewearer after object identification, such as a user's name when the faceis identified). In various examples, the sensory information may beunrelated to the environment, such as providing an incoming text messagevisually or aurally. In various applications, combinations of sensoryinformation augmenting the wearer's environment and sensory informationunrelated to the environment are provided to the wearer.

Modern hearing assistance devices, such as hearing aids, typicallyinclude digital electronics to enhance the wearer's listeningexperience. Hearing aids are electronic instruments worn in or aroundthe ear that compensate for hearing losses by specially amplifyingsound. Hearing aids use transducer and electro-mechanical componentsthat are connected via wires to the hearing aid circuitry.

Hearing aid users often find it difficult to understand speech in noisyconditions. Speech understanding in noisy conditions may be improved byproviding the ability to replay audio from the last few seconds ofspeech. However, in such a case, the repeated speech might overlap withongoing live speech. This problem with overlapping speech can be avoidedby running a live speech recognizer on all speech around the hearing aiduser (either all the time, or on-demand) and displaying the text on ascreen such as that of a smartphone. However, this would require theuser to look continuously at the phone's screen to read the recognitionresult.

What is needed in the art is an improved system for isolating thedesired speaker's sound and displaying speech recognition outputcorresponding to the desired speaker's speech in noisy environments.

SUMMARY

Disclosed herein, among other things, are methods and apparatuses forintegration of hearing aids with a multisensory display device (MSD) toimprove intelligibility in noise.

One aspect of the present subject matter relates to a method ofproviding voice audio to a hearing aid, where the voice audiocorresponds to a speaker of interest within a noisy environment, andwhere the speaker of interest is identified using an MSD. A heads-upgraphical user interface is provided for a hearing aid user to identifya speaker of interest. The lip movement patterns of the speaker ofinterest are recorded by the MSD, and voice activity is detected byanalyzing the recorded lip movement patterns. Audio data are recorded inthe noisy environment using one or more microphones in the hearing aidor MSD. Noise reduction is performed on the audio data using the resultsof the voice activity detection, improving the ability of the noisereduction to distinguish the voice audio of the speaker from othersounds within the noisy environment. The noise-reduced voice audio thenmay be transmitted to the hearing aid. If the speaker of interest isalso wearing an MSD, audio may be recorded at the speaker's MSD andtransferred via a wireless protocol to the user's MSD.

One aspect of the present subject matter relates to a method ofproviding a display of text in an user's field of view, where the textis generated by speech recognition for a speaker of interest within anoisy environment, and where the speaker of interest is identified usingMSDs. A heads-up graphical user interface is provided for a user toidentify a speaker of interest. The lip movement patterns of the speakerof interest are recorded by the MSD, and voice activity is detected byanalyzing the recorded lip movement patterns. Audio data are recorded inthe noisy environment using one or more microphones in a hearing aid orMSD. Noise reduction is performed on the audio data using the results ofthe voice activity detection, improving the ability of the noisereduction to distinguish the voice audio of the speaker from othersounds within the noisy environment. Speech recognition may be performeddirectly on the recorded audio, or preferably on the noise-reduced voiceaudio, and the speech recognition text may be transmitted to anddisplayed on the MSD. This speech recognition and display of text may beperformed on-demand, or may be performed continuously while the user iswearing an MSD. Other aspects are provided without departing from thescope of the present subject matter.

This Summary is an overview of some of the teachings of the presentapplication and not intended to be an exclusive or exhaustive treatmentof the present subject matter. Further details about the present subjectmatter are found in the detailed description and appended claims. Thescope of the present invention is defined by the appended claims andtheir legal equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 and 1B are example variations of MSD form factors according toone embodiment of the present subject matter.

FIG. 2 is an example GUI selection of a speaker of interest according toone embodiment of the present subject matter.

FIGS. 3A and 3B are example flowcharts of the selection of a speaker ofinterest according to one embodiment of the present subject matter.

FIG. 4 is a flow chart of voice activity detection noise suppressionaccording to one embodiment of the present subject matter.

FIG. 5 is an example textual display according to one embodiment of thepresent subject matter.

DETAILED DESCRIPTION

The following detailed description of the present subject matter refersto subject matter in the accompanying drawings that show, by way ofillustration, specific aspects and embodiments in which the presentsubject matter may be practiced. These embodiments are described insufficient detail to enable those skilled in the art to practice thepresent subject matter. References to “an,” “one,” or “various”embodiments in this disclosure are not necessarily to the sameembodiment, and such references contemplate more than one embodiment.The following detailed description is demonstrative and not to be takenin a limiting sense. The scope of the present subject matter is definedby the appended claims, along with the full scope of legal equivalentsto which such claims are entitled.

The present subject matter relates, among other things, to methods andapparatuses for integration of hearing aids with an MSD to improveintelligibility in noisy environments. In various embodiments, thepresent subject matter may include providing voice audio to a hearingaid user, providing speech recognition text to an MSD user, or providingboth voice audio and speech recognition text. The voice audio or textmay correspond to a speaker of interest within a noisy environment, andthe speaker of interest may be identified using MSDs.

FIGS. 1A and 1B are example variations of MSD form factors 100 accordingto one embodiment of the present subject matter. The MSD may includewearable glasses that include side-arms 110. The example MSD in FIG. 1Aincludes frames with lenses 115 a, whereas the example MSD in FIG. 1Bincludes a frame without lenses 115 b. The MSD may include an on-boardcomputing system 120. The MSD may include a microphone 125, which may bean omnidirectional microphone or one or more directional microphones.The MSD may include an image capture device, which may be a still imagecapture device or a video capture device. The MSD may include a displayscreen 135 in the wearer's field of view. The display screen 135 may betransparent, and when used, text or images may be displayed.

FIG. 2 is an example GUI selection 200 of a speaker of interestaccording to one embodiment of the present subject matter. In variousembodiments, an MSD graphical user interface is provided for a user toidentify a speaker of interest. The example selection 200 may occurwithin a portion of the display screen 135 of the MSD. The userinitiates the graphical user interface to identify the speaker ofinterest, and the MSD generates an identification icon. The user mayinitiate the GUI selection of the speaker of interest by pressing abutton on the MSD, by voice command, or by other input. Once the userinitiates selection, an identification icon 210 may be placed in thedisplay screen 135, where the identification icon 210 correspondsapproximately to the center of the user's field of view. Theidentification icon may be in the form of a reticle, crosshairs, circle,or box, or other form. After initiation of the GUI selection, the MSDmay capture and process an image viewable by the MSD. The graphical userinterface may perform preliminary voice activity detection on thecaptured image to identify potential speakers of interest. The user mayaim the identification icon 210 toward one or more speakers of interest.Aiming the identification icon 210 may be performed by the user movinghis or her head, may be performed by providing speech commands or otherinput to move the reticle on the screen, or may be performed by othermeans. During aiming, the identification icon 210 may remain in a fixedposition within the display screen 135, or the reticle may move (e.g.,snap) the identification icon 210 to superimpose the identification icon210 on a specific speaker of interest identified through facialdetection. The example selection 200 may also identify one or morepersons within the display screen 135, where additional persons may bedesignated using additional reticles. When multiple persons areidentified, the user may indicate the speaker of interest by aiming theMSD so the central reticle is superimposed on the desired speaker ofinterest, and providing an input. Also when multiple persons areidentified, the user may provide an input to cycle through identifiedpersons and select the speaker of interest, where cycling may includinghighlighting of each reticle by a change in the reticle size, shape,line weight, or other reticle characteristic.

FIGS. 3A and 3B are example flowcharts 300 of the selection of a speakerof interest according to one embodiment of the present subject matter.FIG. 3A is an example flowchart of the selection of a speaker ofinterest using a single-reticle embodiment, whereas FIG. 3B is anexample flowchart of the selection of a speaker of interest using amultiple-reticle embodiment. In various single-reticle selectionembodiments, the user may initiate 310 the selection of the speaker ofinterest by pressing a button on the MSD, by voice command, or by otherinput. Once the user initiates selection, a central reticle is displayed315 in the display screen, where the reticle corresponds approximatelyto the center of the user's field of view. The user may aim 320 the MSDtoward the speaker of interest, and then provide an input 325 toindicate the speaker of interest is currently within the reticle. Invarious multiple-reticle selection embodiments, the user may initiate350 the selection of the speaker of interest. Additional persons may bedesignated on the MSD by superimposing 355 reticles on each potentialspeaker of interest. To select between or among multiple reticles, theuser may aim 360 a central reticle on the speaker of interest, or theuser may provide an input 365 to cycle through a highlighting of theavailable reticles. The user may provide an input 370 to indicate thespeaker of interest is within the central reticle, or to indicate thespeaker of interest corresponds to the currently highlighted reticle.

FIG. 4 is a flow chart 400 of voice activity detection noise suppressionaccording to one embodiment of the present subject matter. Uponinitiating the GUI 405, the MSD may begin analyzing audio 410, analyzingvideo 415, or prompting for and receiving 420 a user selection of aspeaker of interest. Audio analysis 410, video analysis 415, andreceiving 420 a user selection may occur simultaneously, sequentially,or in any other order. Audio analysis 410, video analysis 415, andreceiving 420 a user selection capability may be built into the MSD, ormay be enabled by sending and receiving audio or video to or from anetwork or another computing device.

The audio analysis 410 may include characterizing whether one or morespeakers may be identified, characterizing the noise level of anenvironment, identifying speakers based on voice patterns, or otheraudio analysis. In various embodiments, a microphone is configured in ahearing aid or MSD such that sound from the environment is received,processed in some way, and played to the hearing aid or MSD wearer.Audio data are recorded in the noisy environment, either using one ormore microphones in the hearing aid or MSD. Audio data may be recordedfor the entire environment using an omnidirectional microphone. Audiodata may be recorded using a directional microphone, where thedirectional microphone may be continuously focused in a directioncoinciding with a reticle. The directional microphone may be in a fixedorientation when using a fixed reticle. The directional microphone maybe adaptively focused when using a moveable reticle, where the focus ofthe microphone array may be steered in a direction (e.g., beamforming)corresponding to the direction of the moveable reticle. Using anomnidirectional microphone or multiple directional microphones, one ormore reticles could snap to one or more speakers of interest, and mayprompt the MSD user to change the speaker of interest. When changing thespeaker of interest, the MSD may snap one or more reticles to speakersof interest, but should minimize the number of unexpected changes, forexample, by focusing on a single speaker of interest until the userprovides an input to change the speaker of interest.

The video analysis 415 may include identifying one or more potentialspeakers of interest through facial detection, performing facialrecognition, performing preliminary voice activity detection throughanalysis of lip movements, or other video analysis. Lip movements mayindicate when the speaker of interest is speaking, and a lack of lipmovement may indicate when the speaker of interest is silent. The lipmovement may be analyzed by a voice activity detection module toidentify the time during which one or more speakers of interest arespeaking. The speaking time may be used to isolate sounds within a noisyenvironment that correspond to the speaker of interest. Video analysis415 may apply a basic speech recognition algorithm on lip movements toprovide alignment of basic sounds (e.g., phonemes). By using timealignment of basic sounds, the algorithm may isolate audio data from thespeaker of interest from other non-speech audio data.

Noise reduction 430 may be performed on the audio data using the resultsof the voice activity detection, improving the ability of the noisereduction 430 to distinguish the voice audio of the speaker from othersounds within the noisy environment. Voice activity detection data maybe used to identify the time during which the speaker of interest isspeaking Using the time during which the speaker of interest isspeaking, the noise reduction 430 may identify voice or speechcharacteristics corresponding to the speaker of interest. Various voiceor speech characteristics of the voice of the speaker of interest may bestored, enabling the MSD to generate a noise reduction profilecorresponding to the speaker of interest.

These audio characteristics may include a frequency range, a voicepattern, or other characteristics. Using identified audiocharacteristics, the noise reduction 430 may diminish voices or othersounds that do not correspond to the speaker of interest. In variousembodiments, MSDs are configured to monitor noise level in theenvironment and apply gain and/or compression to the audio from theaugmented MSD to place spectral level of the MSD audio above thespectral level of the environmental noise, maintaining intelligibilityand sound quality with changing environmental sounds. There may befeedback issues that require a feedback canceller, where the feedbackcanceller is not normally included in such systems. Feedback issues mayhave various causes, such as if the MSD or hearing aid wearer hashearing loss and the audio augmentation includes amplification tocorrect for the hearing loss.

In various embodiments, the noise-reduced voice audio then may betransmitted to and reproduced 440 by the MSD or hearing aid. Thenoise-reduced voice audio may be stored in the MSD or hearing aid forarchival or on-demand audio replay purposes. For on-demand audio replay,the user may provide an input that plays a specific replay duration ofthe noise-reduced voice audio. The replay duration may be configured bythe graphical user interface, or may be preset to replay the previousfive seconds of noise-reduced voice audio.

In various embodiments, text may be generated 450 using speechrecognition performed on the noise-reduced voice audio. The speechrecognition text may be transmitted to and displayed 455 on the MSD. Thespeech recognition text may be displayed instantly on the MSD for theuser, or the speech recognition text may be stored in the MSD forarchival or on-demand text replay purposes. For on-demand text replay,the user may provide an input that plays a specific amount of the speechrecognition text. The amount of speech recognition text may beconfigured by the graphical user interface, or may be preset to replaythe previous sentence of speech recognition text.

FIG. 5 is an example textual display 500 according to one embodiment ofthe present subject matter. In one embodiment, the display screen 135 ofthe MSD may display speech recognition text 510 corresponding to thespeaker of interest. If the MSD has stored one or more noise reductionprofiles, the textual display 500 may indicate an alternate speaker ofinterest is currently speaking 520, and may prompt the MSD user toselect the alternate speaker of interest. The textual display 500 mayindicate an alternate speaker of interest is not currently speaking 530.In some embodiments, a noise reduction profile may be associated withcontact information within a user's list of contacts, and the textualdisplay 500 may indicate the name of the speaker. In some embodiments,the MSD may apply facial recognition software to the speaker ofinterest, and may suggest associating the speaker of interest with oneor more contacts within the user's list of contacts.

In various embodiments, receiver-in-canal hearing aid device (RIC HA)cable assemblies may be attached to an augmented reality MSD to provideaudio delivery with a speaker in the canal, where the in-canal speakeris an alternative to bone-conduction delivery or traditional earphoneinserts. Wireless in-the-canal devices similar to wirelesscompletely-in-the-canal (CIC) hearing aids or a wireless personalamplification device, such as the AMP device, may be configured to pickup sound transmitted from the MSD for the sound delivery system to theMSD wearer. In various embodiments, the wireless transmission could behigh frequency such as 900 MHz or 2.4 GHz. In various embodiments, thewireless transmission may be near field magnetic induction, or may beelectromagnetic signals such that a traditional telecoil or GMRtransducer found in hearing aids can receive the transmitted audio fromthe MSD. In various embodiments, combinations of these systems may beemployed and may be combined with other communication systems.

Some MSDs have bone conduction speakers/transducers to provide audio tothe wearer of the MSD. There are many drawbacks to this, including poorsound quality due to the bandpass nature of sound conducted through theskull, and poor spatial perception due to distortion of binaural cuesnecessary for spatial hearing. In various embodiments, the MSD design isimproved by combining the bone-conducted sound with delivery ofair-conducted sound, such as the provided by a receiver in the wearer'scanal that gets the audio signal from the MSD in a wired or wirelessfashion. The air-conducted sound would enhance the sound quality and/orspatial character of the presented sound while maintaining the benefitsof bone-conduction sound presentation, such as privacy of what is beingheard. Since the air-conducted sound does not have to provide the fullauditory experience, the levels and bandwidth of the air-conducted soundcan be less than what they would have to be if there were no additionalbone-conduction sounds being provided.

In various embodiments, BTE-style cases and hearing aid-type ofelectronics may be physically attached to the MSD so that the audiosignal is electronically passed to the BTE device and the BTE device isresponsible for providing delivery of sound to the wearer.

In various embodiments, hearing aid circuitry is embedded in the MSD toprovide audio that compensates for the hearing loss of the wearer, usingsuch signal processing systems as linear gain, frequency shaping,multiband compression, frequency translation, frequency compression, andcombinations of these.

A deep-fitting device that sits near the ear drum and can stay in aperson's ear canal for weeks without removal and can be used as thesound delivery system for the MSD, where the MSD transmits a digital oranalog audio signal to the deep-fitting device wirelessly.

It is understood that variations in communications circuits, protocols,antenna configurations, and combinations of components may be employedwithout departing from the scope of the present subject matter. Hearingassistance devices typically include an enclosure or housing, amicrophone, hearing assistance device electronics including processingelectronics, and a speaker or receiver. It is understood that in variousembodiments the receiver is optional. Antenna configurations may varyand may be included within an enclosure for the electronics or beexternal to an enclosure for the electronics. Thus, the examples setforth herein are intended to be demonstrative and not a limiting orexhaustive depiction of variations.

It is further understood that a variety of hearing assistance devicesmay be used without departing from the scope and the devices describedherein are intended to demonstrate the subject matter, but not in alimited, exhaustive, or exclusive sense. It is also understood that thepresent subject matter can be used with devices designed for use in theright ear or the left ear or both ears of the wearer.

It is understood that hearing aids typically include a processor. Theprocessor may be a digital signal processor (DSP), microprocessor,microcontroller, other digital logic, or combinations thereof. Theprocessing of signals referenced in this application can be performedusing the processor. Processing may be done in the digital domain, theanalog domain, or combinations thereof. Processing may be done usingsubband processing techniques. Processing may be done with frequencydomain or time domain approaches. Some processing may involve bothfrequency and time domain aspects. For brevity, in some examples mayomit certain modules that perform frequency synthesis, frequencyanalysis, analog-to-digital conversion, digital-to-analog conversion,amplification, and certain types of filtering and processing. In variousembodiments, the processor is adapted to perform instructions stored inmemory that may or may not be explicitly shown. Various types of memorymay be used, including volatile and nonvolatile forms of memory. Invarious embodiments, instructions are performed by the processor toperform a number of signal processing tasks. In such embodiments, analogcomponents may be in communication with the processor to perform signaltasks, such as microphone reception, or receiver sound embodiments(i.e., in applications where such transducers are used). In variousembodiments, different realizations of the block diagrams, circuits, andprocesses set forth herein may occur without departing from the scope ofthe present subject matter.

The present subject matter is demonstrated for hearing assistancedevices, including hearing aids, including but not limited to,behind-the-ear (BTE), receiver-in-canal (RIC), andcompletely-in-the-canal (CIC) type hearing aids. It is understood thatbehind-the-ear type hearing aids may include devices that residesubstantially behind the ear or over the ear. Such devices may includehearing aids with receivers associated with the electronics portion ofthe behind-the-ear device, or hearing aids of the type having receiversin the ear canal of the user, including but not limited toreceiver-in-canal (RIC) or receiver-in-the-ear (RITE) designs. Thepresent subject matter can also be used with in-the-ear (ITE) andin-the-canal (ITC) devices. The present subject matter can also be usedin hearing assistance devices generally, such as cochlear implant typehearing devices and such as deep insertion devices having a transducer,such as a receiver or microphone, whether custom fitted, standard, openfitted, or occlusive fitted. It is understood that other hearingassistance devices not expressly stated herein may be used inconjunction with the present subject matter.

This application is intended to cover adaptations or variations of thepresent subject matter. It is to be understood that the abovedescription is intended to be illustrative, and not restrictive. Thescope of the present subject matter should be determined with referenceto the appended claims, along with the full scope of legal equivalentsto which such claims are entitled.

What is claimed is:
 1. A method comprising: providing a graphical userinterface in a user field of vision for a user to identify and select aspeaker of interest from a plurality of speakers; recording a pluralityof lip movement patterns of the speaker of interest; generating voiceactivity detection data from the plurality of lip movement patterns;recording voice audio data; and generating noise-reduced voice audiodata, wherein generating the noise-reduced voice audio data uses thevoice activity detection data to isolate voice audio within the recordedvoice audio data corresponding to the speaker of interest.
 2. The methodof claim 1, wherein recording a plurality of lip movement patterns ofthe speaker of interest includes steering a focus of a microphone arrayto a location corresponding to the speaker of interest.
 3. The method ofclaim 1, further comprising: transmitting the noise-reduced voice audiodata to a hearing aid worn by the user.
 4. The method of claim 3,wherein the noise-reduced voice audio data is transmitted continually tothe hearing aid.
 5. The method of claim 3, wherein transmitting thenoise-reduced voice audio data to the hearing aid is initiated by theuser.
 6. The method of claim 5, wherein the noise-reduced voice audiodata is stored in an audio data buffer, and wherein a predeterminedduration of stored noise-reduced voice audio data is transmitted to thehearing aid worn by the user.
 7. The method of claim 1, furthercomprising: generating speech recognition text using the noise-reducedvoice audio; displaying the speech recognition text on the graphicaluser interface.
 8. The method of claim 7, wherein the speech recognitiontext is displayed continually on the graphical user interface.
 9. Themethod of claim 7, wherein displaying the speech recognition text on thegraphical user interface is initiated by the user.
 10. The method ofclaim 9, wherein the speech recognition text is stored in a text buffer,and wherein a predetermined amount of stored speech recognition text isdisplayed on the graphical user interface.
 11. The method of claim 1,wherein providing a graphical user interface includes providing auser-worn personal computer.
 12. The method of claim 1, whereinproviding a graphical user interface includes providing a user-worninput device.
 13. The method of claim 1, wherein providing a graphicaluser interface includes providing a gesture sensing input.
 14. Themethod of claim 1, wherein providing a graphical user interface includesproviding a speech recognition input.
 15. The method of claim 1, whereinproviding a graphical user interface includes providing for patientinput to adjust the settings in real-time.
 16. A system comprising: asmart glasses graphical user interface configured to receive a userinput to identify and select a speaker of interest from a plurality ofspeakers; a video recording module configured to record a plurality oflip movement patterns of the speaker of interest; a voice activitydetection module configured to receive the plurality of lip movementpatterns and generate voice activity detection data from the pluralityof lip movement patterns; a voice audio data recording module; and anoise-reduced voice audio data generation module configured to receivethe voice activity detection data and voice audio data, and configuredto generate noise-reduced voice audio data corresponding to the speakerof interest.
 17. The system of claim 16, further comprising: adirectional microphone array configured to focus audio detection in adirection corresponding to the speaker of interest.
 18. The system ofclaim 16, further comprising: a hearing assistance device configured toreceive noise-reduced voice audio data and reproduce audiblenoise-reduced voice audio corresponding to the speaker of interest. 19.The system of claim 18, further comprising: an audio data bufferconfigured to store a predetermined duration of voice audio data,wherein the smart glasses graphical user interface is further configuredto receive a user input to cause the hearing assistance device toreproduce the predetermined duration of audible noise-reduced voiceaudio.
 20. The system of claim 16, further comprising: a speechrecognition text generation module configured to receive thenoise-reduced voice audio data and generate speech recognition text; anda smart glasses graphical user display configured to receive and displayspeech recognition text corresponding to the speaker of interest. 21.The system of claim 17, further comprising: a speech recognition textbuffer configured to store a predetermined amount of speech recognitiontext, wherein the smart glasses graphical user interface is furtherconfigured to receive a user input to cause the smart glasses graphicaluser display configured to receive and display the predetermined amountof speech recognition text.
 22. The system of claim 18, wherein thehearing assistance device includes a hearing aid.
 23. The system ofclaim 18, wherein the hearing aid includes an in-the-ear (ITE) hearingaid.
 24. The system of claim 18, wherein the hearing aid includes abehind-the-ear (BTE) hearing aid.
 25. The system of claim 18, whereinthe hearing aid includes an in-the-canal (ITC) hearing aid.
 26. Thesystem of claim 18, wherein the hearing aid includes a receiver-in-canal(RIC) hearing aid.
 27. The system of claim 18, wherein the hearing aidincludes a completely-in-the-canal (CIC) hearing aid.
 28. The system ofclaim 18, wherein the hearing aid includes a receiver-in-the-ear (RITE)hearing aid.